CT-Based Deep Learning Enables Postoperative Recurrence Prediction for Intrahepatic Cholangiocarcinoma

Preoperatively accurate evaluation of risk for postoperative recurrence contributes to maximizing the therapeutic success of surgical resection for intrahepatic cholangiocarcinoma (iCCA). This study aimed to investigate the potential of deep learning (DL) algorithms for predicting postoperative early recurrence through the use of preoperative images. We collected the dataset, including preoperative plain computed tomography (CT) images, from 41 patients undergoing curative surgery for iCCA at multiple institutions. We built a CT patch-based predictive model using a residual convolutional neural network and used 5-fold cross-validation. The prediction accuracy of the model was analyzed. We de�ned early recurrence as recurrence within a year after surgical resection. Of the 41 patients, early recurrence was observed in 20 (48.8%). A total of 71,081 patches were extracted from the entire segmented tumor area of each patient. The average accuracy of the ResNet model for predicting early recurrence was 98.2% for the training dataset. In the validation dataset, the average sensitivity, speci�city, and accuracy were 97.8%, 94.0%, and 96.5%, respectively. Furthermore, the area under the receiver operating characteristic curve was 0.994. Our CT-based DL model exhibited high predictive performance in projecting postoperative early recurrence, proposing a novel insight into iCCA management.


Introduction
Primary liver cancer was the third leading cause of cancer death worldwide in 2020, with 830,180 deaths contributing to 8.3% of worldwide cancer-related deaths.In 2020, 905,677 liver cancer diagnoses were made globally, comprising 4.7% of worldwide cancer cases 1 .Intrahepatic cholangiocarcinoma (iCCA) is the second most common primary liver cancer after hepatocellular carcinoma and accounts for 10-15% of primary liver cancer 1 .Incidences of iCCA have been increasing over the last three decades 2,3 .
However, the prognosis of iCCA, unfortunately, remains extremely poor, with a 5-year overall survival of 9% 4 .This cancer presents a substantial health problem worldwide, so treatments to improve survival are urgently needed.
In tackling this lethal disease, surgical resection has been the most fundamental and only treatment with the potential for cure 2,5 .Unfortunately, only about 20-40% of patients present with potentially operable disease 3,6 .In addition, recurrence following surgical resection of iCCA remains a big concern.Several studies have demonstrated that around 50% of patients recurred after curative-intent surgery 5,7,8 .In short, the number of patients who bene t from surgical resection alone is limited.
Preoperatively accurate evaluation of risk for postoperative recurrence contributes to maximizing the therapeutic success of surgical resection.Several lines of evidence from the clinical studies have demonstrated that factors associated with a higher risk for recurrence include large tumor size, multiple tumors, vascular invasion, lymph node metastasis, and R1 resection [7][8][9] .These reports provided us with signi cant insight.However, de nitive preoperative diagnosis of the above factors, especially lymph node metastasis, remains challenging 10 .Furthermore, these reports were analyzed using multivariable logistic regression modeling to identify the independent risk factors for postoperative recurrence.This method has been traditionally performed in clinical studies, but there have been certain limitations, such as selection of variables, confounding factors, and multicollinearity.
To resolve the above issues, in this study, we created a prediction model for postoperative recurrence using arti cial intelligence (AI).AI has the potential to revolutionize disease diagnosis and management in the medical eld 11 .Deep learning (DL) has recently gained extensive attention as a technique for realizing the full potential of AI 12 .Convolutional neural networks (CNNs), which are a DL approach, are especially recognized as demonstrating high performance in image recognition 13,14 .Indeed, there have been some successful reports in applying DL to the assessment and prediction of radiological images in clinical settings 13,15,16 .
In the eld of iCCA, a recent study using DL has demonstrated the feasibility of applying it to liver tumor diagnoses 17 .Moreover, Jeong et al. showed its usefulness in the prognostic estimation and strati cation of susceptible individuals for adjuvant treatment after resection 18 .However, there has been no report applying DL to recurrence prediction in patients with iCCA after resection.Thus, this study aimed to investigate the potential of DL algorithms for predicting postoperative recurrence through the use of preoperative images of iCCA.Here, we have successfully developed a prediction model using CNNs and propose a novel concept in iCCA management from a completely different perspective.

Comparison of the perioperative characteristics of the nonearly recurrence and early recurrence groups
The clinical characteristics of the 41 enrolled patients are shown in Table 1.Of the 41 patients, early recurrence was observed in 20 (48.8%).The early recurrence group demonstrated increased levels of tumor biomarkers such as carbohydrate antigen 19 − 9 and carcinoembryonic antigen (Table 1).However, there were no signi cant differences in the biomarkers between the groups.Univariate analysis indicated that Union for International Cancer Control (UICC) N category was the only signi cant predictor of early recurrence (P = 0.002).The odds ratio (OR) was 11.611 (95% con dence interval (CI): 2.116-63.726).

Binary logistic regression analysis
To predict early recurrence, we performed a binary logistic regression analysis, which is one of the traditional methods.We set early recurrence as the dependent variable.N category, which was found as a signi cant predictor through univariate analysis, was entered into a binary logistic regression analysis.
Furthermore, we entered the UICC T category and surgical margin status, which have been reported as factors associated with a higher risk for recurrence.

Performance of the ResNet50 model
A total of 25,765 patches were obtained from the 21 patients without early recurrence.Furthermore, a total of 45,316 patches were obtained from the 20 patients with early recurrence after surgery.Finally, a total of 71,081 patches were extracted from the 41 patients in the current study.3).The model achieved an AUC of 0.994 (95% CI: 0.993-0.995) in the validation dataset (Fig. 2).

Highlighting areas with the risk of early recurrence by heatmap
A representative heatmap of the tumor area on a preoperative plain CT based on probability calculated using our prediction model is shown in Fig. 3.The heatmap can be superimposed on the input image to highlight the areas the model considers important in making its diagnosis.In short, the heatmap can contribute to assisting physicians before surgery by highlighting areas with the risk of early recurrence.

Factors that can in uence misprediction
We further investigated factors that can in uence misprediction.Using our own model, we calculated the prediction accuracy of each case based on the patches in each one.In this study, we de ned cases with prediction accuracy of the rst quartile or less (96.0%), as occurrences of misprediction.Table 4 reveals a comparison of the perioperative characteristics of the cases with an accuracy of 96.0% or less and the cases with over 96.0%accuracy.Univariate analysis showed that smaller tumor size was a signi cant factor in misprediction (P = 0.025).Likewise, looking at tumor size only, also showed a signi cant difference in the comparison of the two groups when divided by the median value of prediction accuracy (97.8%).Collectively, these data suggested that our prediction model can perform excellently, particularly with larger iCCA.

Discussion
We applied the DL model to predict postoperative early recurrence of iCCA.We have successfully demonstrated high performance in the prediction of postoperative recurrence using plain preoperative CT images.The accuracy of the DL model far exceeded that of the binary logistic regression analysis (AUC, 0.994 vs. 0.770).This report represents the rst study in which a DL model based on CT images is used to predict postoperative recurrence in iCCA.Our results may yield a novel insight into personalized treatment strategies, including neoadjuvant and adjuvant chemotherapy, in iCCA management.
Isolating results has been a challenge as past prospective randomized trials have included not only iCCA but also other bile duct cancers [21][22][23] .Furthermore, the indication for adjuvant chemotherapy in those studies was heterogeneity.In short, the selection criteria of susceptible individuals for adjuvant chemotherapy is not well established.To address this issue, Jeong et al. showed the usefulness of an AI framework in the prognostic estimation and strati cation of susceptible individuals for adjuvant treatment after resection in iCCA patients.In contrast, we intended to predict postoperative recurrence directly.Our model, which can directly predict early recurrence, would be used to predict who should receive adjuvant chemotherapy based on their risk of recurrence.
Liang et al. conducted a single-center retrospective study and built a radiomics nomogram to predict early recurrence of iCCA after surgical resection 24 .Their nomogram, using preoperative arterial-phase contrastenhanced magnetic resonance imaging (MRI), achieved an AUC of 0.82 and 0.77 in the training and validation cohorts, respectively.They used manual engineered features and selected the earlies recurrence-related features using a least absolute shrinkage and selection operator logistic regression analysis.Zhao et al. used radiomics from MRI to predict early recurrence.Their radiomics model showed a preferable predictive performance (AUC 0.889) 25 .Compared with the previous radiomics model using MRI, our model, which is based on DL features, achieved higher predictive performance (AUC 0.994).
Based on our results, which perform in such a highly predictive manner with the model addressing postoperative early recurrence, we propose a new concept in iCCA management.Though we need to discuss further which population, patients, those with or without early recurrence, is t for adjuvant chemotherapy, achieving quite high levels of predictive accuracy, compared to conventional methods, can provide valuable information for determining adjuvant therapy and developing surgical plans, thereby facilitating pretreatment decisions.
Thanks to the advantages reaped from DL, we physicians, can easily apply computer-aided diagnosis 16,26 .Deep learning algorithms, such as CNN, have been widely used in the eld of image diagnosis and prediction owing to their being fast, accurate, and reproducible 26,27 .CNN can uncover details in medical images that human experts cannot nd, and automatically render a quantitative assessment 28 .Generally, even expert radiologists and surgeons cannot always access meaningful ndings that would enable physicians to decide on a treatment strategy from plain CT images.In fact, there have been no reports or guidelines that recommend using plain CT images for risk assessment of postoperative recurrence in iCCA.Several lines of evidence, including our study, can lead to a paradigm shift in the recognition of AI in the eld of iCCA treatment.
The present study has several limitations.This is a retrospective study.In addition, although this is a pilot study, the patient population was small.However, our model achieved high predictive performance.If we had access to additional training data from a large cohort, we could achieve even higher prediction accuracy and generality.To establish clinical applications, su cient datasets are fundamental requirements.A novel AI approach based on analyzing a huge database, such as national or regional datasets, would be attractive to both clinicians treating iCCA and their patients.An accurate and robust prediction model can ultimately contribute to a better prognosis in iCCA patients.
There was the question of possible lack of homogeneity in CT techniques over the past 20 years that has been a point of contention (Supplemental Table 1).Nevertheless, our model achieved high predictive performance.These results suggested that relative heterogeneity of CT techniques may not be a big issue because of the handling of huge information from CT images through DL.Certainly, homogeneity of CT techniques would be preferable.However, it would not be practical in a real clinical setting for all patients to undergo CT exams using the same scanner and technique.In short, the use of a diverse set of CT acquisitions was not a limitation, it was a bene t to the study.
In conclusion, our DL model, using plain preoperative CT images of iCCA, exhibited high predictive performance in projecting postoperative early recurrence.The present multicenter study has provided a novel approach to predict recurrence after surgery.This model may help clinicians in the selection of patients for neoadjuvant and/or adjuvant therapy.This approach can contribute to personalized strategies in iCCA treatment.To establish a clinical application, conducting a study using a huge dataset, such as national dataset, is the hope for the future.

Patients and study design
This multi-institutional, retrospective, observational study was approved by the Committee of Medical Ethics of Hirosaki University Graduate School of Medicine (Aomori, Japan; reference no.2020-230-1).
Informed consent was obtained in the form of opt-out on our website (https://www.med.hirosakiu.ac.jp/hospital/outline/resarch/resarch.html), with the approval of the Committee of Medical Ethics of Hirosaki University Graduate School of Medicine.Our study did not include minors.This study was designed and carried out in accordance with the Declaration of Helsinki.
Between January 2001 and December 2019, 41 patients undergoing liver surgery for iCCA at three institutions were included in this study.All patients had a con rmed pathologic diagnosis.In this study, the exclusion criteria were as follows: patients who had not undergone CT examination before surgery or patients with missing postoperative course data.Recurrence after surgery was diagnosed using enhanced CT and/or MRI.In this study, early recurrence was de ned as recurrence within a year after liver surgery.

CT acquisition and tumor segmentation
Our work ow is shown in Fig. 1.Preoperative axial plain CT images for each case were obtained from multiple institutions and were used for this study.Radiological assessment was performed by boardcerti ed radiologists who were blind to the outcome of the patients.Board-certi ed surgeons and medical students performed CT acquisition and tumor segmentation based on the radiological assessment.Using a commercial viewer (ShadeQuest/ViewR, Fuji lm, Japan), the CT image showing the largest tumor area was selected.The entire tumor region was manually segmented with Adobe Illustrator and saved.

Preparation of dataset
We trimmed a patch a size 128 x 128 pixels with 32 pixel stride from the entire segmented tumor area.Finally, 71,081 patches were obtained from 41 patients in the current study.
Architecture of the CNN

Evaluation methods
We cross-validation to obtain more accurate results with less bias in the machine learning studies 30 .In this study, the dataset is split into ve folds, one fold of which is for validation and the other folds are for training.The proportion of patients with versus without early recurrence was equal in each fold.
The training and validation processes were repeated ve times using different folds each time.The nal results were then averaged and the standard deviation was calculated.The accuracy, sensitivity, speci city, positive predictive values, and negative predictive values were evaluated.The model was also evaluated using the area under the receiver operating characteristic (ROC) curve (AUC).
Heatmap probability of early recurrence of the patches calculated by the trained model over the entire tumor area was modulated in gray scale from 0 to 255, which was assigned to pseudo-coloring; blue for a lowrisk patch and red for a high-risk one on CT imaging.Heatmaps were generated by applyColorMap in OpenCV to visualize the most indicative region for the risk of early recurrence.Prediction of the probability of early recurrence was visualized by pseudo-coloring; COLORMAP_JET was applied to the grayscale CT images.

Statistical analyses
Continuous variables were expressed as medians (ranges) and analyzed using nonparametric methods for non-normally distributed data (Mann-Whitney U-test).Categorical variables were reported as numbers (percentages) and analyzed using the chi-squared test or Fisher's exact test, as appropriate.Variables with a signi cant relationship to early recurrence in univariate analysis were used in a binary logistic regression model.A difference was considered to be signi cant for values of P < 0.05.The statistical analyses were performed using IBM SPSS Statistics for Windows, Version 26.0 (IBM Corp, Armonk, NY, USA).

Figures
Page 18/  The study work ow and methodological process.

Figure 2 The
Figure 2

Table 1
Comparison of the perioperative characteristics of the non-early recurrence and early recurrence groups.

Table 2
Performance of the DL model in the training data set.
The average accuracy of the ResNet model for predicting early recurrence was 98.2% for the training dataset.The average sensitivity, speci city, and positive and negative predictive values were 98.9%, 97.0%, 98.3%, and 98.0%, respectively (Table2).The model achieved an AUC of 0.9983 (95% CI: 0.9982-0.9985) in the training dataset.Likewise, the model showed high predicting performance in the validation dataset.The average sensitivity, speci city, and positive and negative predictive values were 97.8%, 94.0%, 96.7%, and 96.1%, respectively.In the validation dataset, the ResNet model achieved an accuracy of 96.5% (Table

Table 3
Performance of the DL model in the validation data set.

Table 4
Comparison of perioperative characteristics depending on prediction accuracy.